Modelling Program Verification Tools for Software Engineers

In software engineering, models are used for many different things. In this paper, we focus on program verification, where we use models to reason about the correctness of systems. There are many different types of program verification techniques which provide different correctness guarantees. We investigate the domain of program verification tools, and present a concise megamodel to distinguish these tools. We also present a data set of almost 400 program verification tools. This data set includes the category of verification tool according to our megamodel, practical information such as input/output format, repository links, and more. The categorisation enables software engineers to find suitable tools, investigate similar alternatives and compare them. We also identify trends for each level in our megamodel based on the categorisation. Our data set, publicly available at https://doi.org/10.4121/20347950, can be used by software engineers to enter the world of program verification and find a verification tool based on their requirements

Verifying Sanitizer Correctness through Black-Box Learning: A Symbolic Finite Transducer Approach

String sanitizers are widely used functions for preventing injection attacks such as SQL injections and cross-site scripting (XSS). It is therefore crucial that the implementations of such string sanitizers are correct. We present a novel approach to reason about a sanitizer’s correctness by automatically generating a model of the implementation and comparing it to a model of the expected behaviour. To automatically derive a model of the implementation of the sanitizer, this paper introduces a black-box learning algorithm that derives a Symbolic Finite Transducer (SFT). This black-box algorithm uses membership and equivalence oracles to derive such a model. In contrast to earlier research, SFTs not only describe the input or output language of a sanitizer but also how a sanitizer transforms the input into the output. As a result, we can reason about the transformations from input into output that are performed by the sanitizer. We have implemented this algorithm in an open-source tool of which we show that it can reason about the correctness of non-trivial sanitizers within a couple of minutes without any adjustments to the existing sanitizers

Evaluation of Specification Inference Tools for Deductive Verification

About this artifactAs part of our research, we evaluated the impact of several specification inference tools for deductive verification.This artifact contains the following things:- input-output-examples-for-specification-inference/: A directory containing the inputs we used to test specification inference tools as well as the output that the tools ChatGPT, Daikon, EvoSpex, Strongarm and Toradocu generated.- Strongarm.ova: A Virtual Machine with the tool Strongarm installed. It also contains the examples we used to test the tool.- SpecInferenceSurveyArtifact.ova: A Virtual Machine with the tools Daikon, EvoSpex and Toradocu installed. It also contains the code, documentation and a test suite for the examples we used to test the tools.You can use the Virtual Machines to try the specification inference tools yourself.If you're only interested in the input/output we used for testing the tools, then the input-output-examples-for-specification-inference/ directory should be sufficient.License informationThe tools are shared under their original licenses which are included in the dataset.The ArrayList.java and Counter.java examples are licensed under CC-BY 4.0.The BinarySearchGood.java example is originally from https://www.openjml.org/examples and is shared under the CC-BY-NC 4.0 license.</p

Evaluation of Specification Inference Tools for Deductive Verification

About this artifactAs part of our research, we evaluated the impact of several specification inference tools for deductive verification.This artifact contains the following things:- input-output-examples-for-specification-inference/: A directory containing the inputs we used to test specification inference tools as well as the output that the tools ChatGPT, Daikon, EvoSpex, Strongarm and Toradocu generated.- Strongarm.ova: A Virtual Machine with the tool Strongarm installed. It also contains the examples we used to test the tool.- SpecInferenceSurveyArtifact.ova: A Virtual Machine with the tools Daikon, EvoSpex and Toradocu installed. It also contains the code, documentation and a test suite for the examples we used to test the tools.You can use the Virtual Machines to try the specification inference tools yourself.If you're only interested in the input/output we used for testing the tools, then the input-output-examples-for-specification-inference/ directory should be sufficient.License informationThe tools are shared under their original licenses which are included in the dataset.The ArrayList.java and Counter.java examples are licensed under CC-BY 4.0.The BinarySearchGood.java example is originally from https://www.openjml.org/examples and is shared under the CC-BY-NC 4.0 license.</p

Specification Translator: Tool to Translate Specifications for Deductive Verifiers

About the Specification TranslatorThe Specification Translator is a tool that has been implemented as part of our research titled "Join Forces! Reusing Contracts for Deductive Verifiers through Automatic Translation". This tool will translate specifications in verified Java programs from one specification language into another. It supports the tools Krakatoa, OpenJML and VerCors.The tool takes an annotated Java program and a target tool as input. It will then generate an annotated Java program where the annotations have been translated.What's included in this artifact?This artifact contains the following things:- specification-translator.zip: A directory containing the Specification Translator tool as well as the examples used for the evaluation.- SpecTranslatorArtifact.ova: A Virtual Machine with the Specification Translator tool, as well as the OpenJML and VerCors verifiers. It also contains the examples used for the evaluation and a script to reproduce the evaluation.You can use the Virtual Machine to reproduce the evaluation including verification after translation.If you just want to use the Specification Translator or have a look at the input/output files of the evaluation, then the zip file is sufficient.License informationThe Specification Translator tool is shared under the CC-BY 4.0 license.The verifiers in the VM, as well as the examples and case studies used for the evaluation, are not licensed under the CC-BY 4.0 but under their original licenses which have been included.</p

Specification Translator: Tool to Translate Specifications for Deductive Verifiers

About the Specification TranslatorThe Specification Translator is a tool that has been implemented as part of our research titled "Join Forces! Reusing Contracts for Deductive Verifiers through Automatic Translation". This tool will translate specifications in verified Java programs from one specification language into another. It supports the tools Krakatoa, OpenJML and VerCors.The tool takes an annotated Java program and a target tool as input. It will then generate an annotated Java program where the annotations have been translated.What's included in this artifact?This artifact contains the following things:- specification-translator.zip: A directory containing the Specification Translator tool as well as the examples used for the evaluation.- SpecTranslatorArtifact.ova: A Virtual Machine with the Specification Translator tool, as well as the OpenJML and VerCors verifiers. It also contains the examples used for the evaluation and a script to reproduce the evaluation.You can use the Virtual Machine to reproduce the evaluation including verification after translation.If you just want to use the Specification Translator or have a look at the input/output files of the evaluation, then the zip file is sufficient.License informationThe Specification Translator tool is shared under the CC-BY 4.0 license.The verifiers in the VM, as well as the examples and case studies used for the evaluation, are not licensed under the CC-BY 4.0 but under their original licenses which have been included.</p